Journal of Education 
and e-Eearning Research 

ISSN(E) : 2410-9991 
ISSN(P) : 2518-0169 
Vol. 3, No. 3, 78-86, 2016 

httD'J/asianonlineiournals.com/index.DhD/JEELR 



CrossMark 

click for updates 



Examinations Wash Back Effects: Challenges to the 
Criterion Referenced Assessment Model 

Mogapi, M 1 © I Educational Foundations, University of Botswana 


Abstract 

Examinations play a central role in the educational system due to the fact that information generated from 
examinations is used for a variety of purposes. Critical decisions such as selection, placement and 
determining the instructional effectives of a programme of study all depend on data generated from 
examinations. Numerous research studies have shown that examinations have an influence on the learning 
and teaching process and that such influence could be either positive or negative. It is imperative that 
washback effect is evaluated to determine its impact on the quality of learning and teaching that goes on 
in the schools. This paper presents evidence that point towards presence of examination washback 
effect on the teaching process as a result of the format and content of the Primary School 
Leaving Examinations in Botswana. A questionnaire containing quantitative and qualitative items was 
used to capture opinions of teachers relating to the extent to which final examinations influences topic 
select , content coverage, teaching methods, and classroom activities. Generally, opinions presented 
by the teachers point towards presence of a negative washback effect. This apparent negative 
washback effect on the curriculum presents a challenge to a successful implementation of criterion 
reference testing system in Botswana. 

Keywords: Examinations washback effect, Washback hypotheses, Measurement driven instruction, Construct 
underrepresentation. 
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1. Introduction and Background Information 

Assessment plays a central role in any educational system in the sense that data collected through examinations 
provide information needed for making high stake decisions. Valid and reliable scores have to be generated in order 
to determine whether learning is really taking place within the classroom. The relationship between teaching and 
examinations has attracted the attention of researchers such as Ahmed and Rao (2012); Alderson and Wall (1993); 
Buck (1988); Messick (1996); Nkosana (2009); Shohamy (1993) and Shohamy et al. (1996). The interest emanates 
from the apparent influence of examination information on learning and teaching. Extensive research of the 
washback construct culminated in the production of a number of examination washback models. The model 
presented below is by Bailey (1996) and consists of three main components: these are participant, process and 
product. 


i»aH' nt'iPANTS pnnri{S5HS pjtouijcrrs 



The model shows that students and teachers are the main actors within the classroom. However, their actions are 
directly influenced by content sources generated by curriculum designers and other publishers as well as researcher 
results. In educational literature, there is a general consensus that examinations influence teaching and learning; this 
influence is generally referred to as washback (Pizarro, 2010). Alderson and Wall (1993) observed that washback 
effect within the classroom can either be positive or negative. Positive washback occurs in a situation where the 
information from examination clearly indicates areas of strength and weakness for each learner. Teachers can 
therefore use the information to institute necessary intervention strategies aimed at promoting learning in the 
classroom. For example, if examination feedback information indicates high content mastery with respect to handling 
of abstract concept but noticeably low performance in the area of concrete application of the concepts, teachers can 
respond to this shortcoming by modifying their instructional delivery practices to address the identified area of need. 
Hughes (1989) listed procedures that could be followed to achieve positive washback. These include sampling a 
wide area of the content domain and using criterion referenced testing techniques. This is otherwise referred as 
measurement-driven instruction (MDI); that is, using a test as a deliberate washback generating tool (Pizarro, 2010) 
On the other hand, negative washback could be experienced when the nature and content of the examination lead 
teachers to concentrate on topics that regularly appear in the examination. In such a situation, learners are only 
exposed to regularly tested subject matter and do not have the opportunity to acquire relevant skills related to 
objectives not tested. Negative washback effect essentially results in a situation where content in the syllabus is 
significantly trimmed and emphasis is more on memorization on facts rather than assisting learners to develop 
critical thinking skills. This anomalous practice within the classroom environment is captured well by Buck (1988) in 
the following statement; 

However, if the examinations are not testing these goals, students, who usually have their 
sights set on the examination, may pass the tests but totally fail to attain the basic goals set 
by educational planners. In such a case the washback effect is very negative indeed, (p. 17) 

In Botswana, examination washback has been a concern for policy makers and educational practitioners alike. 
For example, an observation was made in the Education for Kagisano report of 1977 about the relationship between 
teaching and examinations at different levels in the system. It is clearly stated in the report that; 

The system emphasizes preparing pupils at each educational level for the major examinations at 
Standard 7, Form III, and Form V; rewarding the competencies that will be used at the next higher 
education level, and ultimately preparing young Batswana for white color, modem sector jobs... This 
creates strong pressures towards memorization of facts in preparation for the exams rather than 
strengthening the basic skills for pupils and developing the ability to think and reason (Republic of 
Botswana, 1977). 

The current paper assesses examination washback effect at primary school level. The relevance of the current 
study is necessitated by the country’s transition from norm referenced testing (NRT) model to criterion referenced 
testing (CRT) practices in 1994. Introduction of criterion referenced testing at primary school level was subsequent 
to the recommendations of the Revised National Policy on Education that were made in 1993. Primary school system 
transformed from being a high stake selection exam to an instrument used for diagnostic purposes. CRT allows for a 
strong linkage between the item in the test and objectives reflected in the syllabus such that performance on the test 
faithfully reveals the knowledge status of the each candidate. Objective based testing was also aimed at making sure 
that there is a seamless connection between assessment at national and classroom level. If there is a seamless 
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connection between the two, then teachers will concentrate more on assisting learners to acquire the skills reflected 
in the syllabus. Feedback information from final examinations will serve to support learning and teaching process 
thus exerting positive washback effect in the system. Classroom activities will be geared towards achievement of 
objectives in the syllabus. Assessment then will functions as a 4 systematic process of gathering evidence of what the 
child can do relative to the instructional outcome, gathered at an opportune time that will allow the teacher to use 
the feedback to make judgment about appropriate adjustments in the instructional environment ’ (Republic of 
Botswana, 1992). 

1.1. Statement of the Problem 

The prevalence of negative washback effect due to the influence of the content and nature of the examination 
will lead to selective teaching and learners will be drilled on content regularly tested in the examination. Important 
subject matter or objectives in the syllabus that are not often tested are ignored. This has the effect of significantly 
narrowing down the content in the syllabus and thus compromising the quality of basic education in Botswana. This 
research study is based on three research questions listed below; the questions are derived from the washback 
hypotheses model proposed by Alderson and Wall (1993). The research questions are: 

To what extent does examination influence the teacher’s content selection? 

To what extent does examination influence the degree or depth of teaching? 

To what extent does examination influence the teacher’s selection of classroom activities? 

1.2. Significance of the Study 

Delineating elements that promote negative washback effects in the learning and teaching environment will 
allow for development and implementation of appropriate intervention strategies. The direct effect of this will be a 
concomitant improvement in the learning and teaching process as teachers will be compelled to cover a broad area of 
the syllabus. The quality of education will subsequently improve as learners will be exposed to a full spectrum of the 
knowledge and skills outlined in the syllabus. 

2. Literature Review 

Andrew et al. (2002) conducted a research study aimed at assessing the effects of introduction of a new test 
within the Hong Kong Advanced Use of English programme. The new test served as a high stakes selection 
instrument for student intending to proceed to tertiary institutions. The test was taken by approximately 20 000 
prospective students each year after their final year of secondary education. The introduction of the test was in 
response to low English language proficiency level exhibited by undergraduates students. Therefore, the new test 
was meant to engender deliberate positive washback effect in the teaching of English language in order to raise the 
language proficiency level of prospective students. The researchers noted that 4 The specific intention was that the 
addition of an oral exam (with a substantial weighting of 18%) would encourage continuing attention, by teachers 
and students, to the speaking skill in the last 2 years of school” (p. 210). The test was administered over a period of 
three years (i.e., 1993, 1994 and 1995) to three different groups. The three separate groups were used to test for the 
effects of washback influence. The data collected was videotaped and subsequently analyzed by trained assessors. 
The researchers observed a significant difference between the 1993 and 1995 groups in individual presentation and 
group discussions. For example, the 1993 group tended to make general introductory remarks that were content 
based whereas the 1995 cohort used more direct format similar in type to the one in the prescribed text books. This 
piece of evidence made the researchers to conclude that the new test had some positive influence on the teaching and 
learning process since the changes associated with the 1995 group wash in line with the demands of the new test. The 
researchers opined that; “Given that title introduction is the only strategy suggested by the textbooks for starting a 
presentation, it seems reasonable to hypothesise that this marked contrast is evidence of washback mediated through 
published materials” (Andrew et al , 2002). 

Cheng (1998) carried out a similar study aimed at investigating the impact of public examination innovation 
introduced by Hong Kong Public Examinations Authority. The expected changes also followed the philosophy of 
positive washback or measurement driven instruction. Positive washback was expected to result in the narrowing 
down of the gap that existed between what happens in the exam room and the outside world (Cheng, 1998). Cheng’s 
study emphasized on perceptions and attitudes of learners as they may relate to changes in test contents. Participants 
in the study were sampled in 1994 and 1995. In 1994, 1100 students were sampled while in 1995 only 600 were 
sampled. The difference in the sample size is ascribed to resource limitations. The 1994 group used the old 
examination format while the 1995 one was treated as an experimental group. Subsequent comparison showed that 
the experimental group used more English in the classroom than the control group. Secondly, the experimental group 
reportedly went for more coaching clinics and did more language games and group discussions. Basing on this 
evidence, the researcher concluded that the examination drove teaching in the direction of coaching and drilling for 
what was required in the exam. Therefore, a significant increase in coaching activities and memorization of content 
indicated presence of negative washback effect. 

In 1996, Alderson and Hamp-Lyons published a paper that was based on their research which sought to 
investigate common claims that TOEFL (Teaching of English as a Foreign Language) was exerting an undesirable 
influence on language teaching. Another criticism against TOEFL was that it raised student scores without really 
equipping them with language skills they will need at tertiary level (Alderson and Hamp-Lyons, 1996). The study 
was based in a specialized language institution in the USA; students from the institution are regularly admitted to 
USA universities. The researchers interviewed students and teachers to record their opinions and attitudes towards 
TOEFL. Specifically, students were asked to give their opinion on how they would like TOEFL preparation courses 
taught. Evidence collected indicated that most teachers had a negative attitude towards TOEFL because they so it as 
being inauthentic and non-communicative. They also expressed the view that it was students who drove the 
methodology as they insisted to be taught in a way that will help them do well in TOEFL test. 
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Another informative washback effect study was done by Tabataba and Estani (2013) in Iran. The purpose of the 
study was to determine the examination washback influence of IELTS Academic Writing Module test on learners’ 
motivation, test taking anxiety, test taking strategies and course expectation. Participants for the study were sampled 
from Advanced Writing and Academic Writing classes; the total number of student selected was 79. Both 
quantitative and qualitative data collection methods were used. For example, questionnaire responses were 
triangulated with evidence from observation and interview sessions. A multivariate test for repeated measures and 
analysis of covariance were employed to tease out significant difference between the groups. The repeated measures 
technique produced non-significant results showing no washback effect. However, the ANOVA analysis indicated 
significant difference between the groups at an alpha level of .05 (Tabataba and Estani, 2013). The researchers were 
able to extract two key points from the study. Firstly, the innovation in the examinations system failed to produce 
desired results. Secondly, washback effect tended to vary from learner to learner. These two results lead the 
researcher to suggest a new washback model that reflected the interactive and multidirectional nature of the 
washback variables. 

An interesting mixed model study was also conducted by Ahmed and Rao (2012) in Pakistan. The main 
objective of the study was to determine the effects of examination washback on the teachers’ selection of 
methodology in English Language teaching in higher secondary schools. A teacher could either use the Grammar 
Translation Method (GMT) or the Communicative Method. Purposive sampling technique was used to identify 128 
high school teachers who were later required to respond to a questionnaire containing Likert scale items. The 
researchers also carried out document analysis by evaluating content in the prescribed text books and examination 
papers. The result of the study showed that teachers preferred to use the Traditional Grammar Translation Method 
(GTM) as opposed to the recommended Communicative method (Ahmed and Rao, 2012). The selection of GTM was 
attributed to the nature and content of the exam which tended to focus more on grammar. 

Salihi (2012) published a paper entitled ‘The washback effect of Iranian Universities Entrance Exam: Teachers’ 
insights’. The paper looked at the influence of Entrance Examination (EEC) test on Iranian high school teachers. 
The EEC is a high stake selection exam used to choose students proceeding to tertiary level education. Stratified 
sampling procedures were applied to ensure fair representation of teachers from all five educational districts in the 
Iranian city of Isfanhen. The target population for the study was an estimated 436 teachers from which a sample of 
200 was taken. The researcher mailed the questionnaire to the selected teachers; the response rate was 66%. 
Subsequent data analysis showed that 89.4% of the teachers indicated that they put more emphasis on reading 
comprehension. Also 74.2% of the teachers said they teach according to EEC format (Salihi, 2012). The conclusion 
made was that the EEC led teacher to ignore communicative approach in favour of mock examinations. 

The University of Victoria in Canada commissioned a study in 2006 to determine ways and means of improving 
the language proficiency of the teaching assistants employed within the campus. The internationalization programme 
of the university led to a substantial increase in the number of foreign students being employed as teaching assistants. 
It became important then to raise the language competency level of students intending to serve as teaching assistants. 
The main objective of the study was to explore the extent to which the new Spoken Language Ability Test could be 
used to stimulate positive washback in the teaching and learning of English language. The initial sample for the study 
comprised 47 registered foreign students; the students were required to take the old version of the Spoken 
Proficiency English A Test or SPEAK. Students who scored 220 out of the 300 items in the test were required to take 
the new version of SPEAK. Successful students were divided into two groups (Control and Experimental Group) and 
put on a 12 week long training programme. The control group continued with the regular programme while the 
experimental group took a language course. The paired sample t test revealed a significant difference in the mean 
values of the two groups. This difference suggested that the new test (i.e., Spoken Language Ability Test) had some 
positive influence on classroom related areas such as teaching content, teaching method, and student learning. 
According to Saif (2006) ‘The test appeared to have directly and intensively influenced the teacher's choice of both 
main and supplementary material used outside and inside the classroom (p. 28). 

In 1998, the assessment at senior secondary school level in Botswana transitioned from Cambridge Oversea 
School Certificate (COSC) to the new Botswana General Certificate of Secondary Education (BGCSE). The change 
was meant to introduce a curriculum that was more relevant to the needs of the country. For example, English 
language under COSC had only two components (i.e., Reading and Writing) whereas under BGCSE all the four 
components (Reading, Writing, Speaking and Listening) were included in the syllabus. However, Listening 
Comprehension was not to be assessed in the final examinations. Nkosana (2009) carried out a study to establish the 
effects of non-examinable content on teaching and learning. A total of 553 students and 53 teachers were sampled 
from 10 secondary schools in Botswana. Both teachers and students responded to a questionnaire containing closed 
and open ended items. The results of the study indicated that both students and teachers regarded listening as a very 
important skill even though it was not assessed. Also, teachers felt inclusion of listening in examination would 
encourage students to take it more seriously (Nkosana, 2009). Essentially, the finding from the study did not support 
the washback hypotheses that examinations will influence what is taught in the classroom. 

The literature on examination washback presented above show a lack of consensus amongst research studies 
done in different countries. Some studies clearly showed examinations have a positive influence on learning and 
teaching (e.g. Spratt (2005); Saif (2006)). Some researchers (e.g., (Cheng, 1998; Saeed and Rao, 2012)) have 
presented evidence that point to the existence of negative washback effect. Scholars like Alderson and Alderson and 
Wall (1993) have even questioned the validity of the washback construct. However, the need to evaluate the impact 
of examinations on the learning and teaching process remains to be important. Results from such studies could be 
used to inform policy formulation and influence day to day classroom practice. 
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3. Methodology 

Both quantitative and qualitative data collection methods were used in the study. Evidence collected from a 
sample of teachers was used to determine the nature of examination washback prevalent in primary school 
classrooms. A questionnaire containing 36 quantitative items and a few qualitative questions was used to capture the 
views and perceptions of teachers regarding the influence of past examinations on teaching. The quantitative items 
were in the form of a Likert scale allowing teacher to indicate whether they strongly agree, agree, disagree or 
strongly disagree with the statement presented. Qualitative questions, on the other hand, generated narrative 
responses in order to directly capture perceptions and opinions of participants. Most of the questionnaires were 
administered during the month of August in 2013. The data was then analyzed using SPSS version 20. 

3.1. Population Characteristics and Sampling Procedures 

Students who registered for a degree programme in the Department of Primary Education (University of 
Botswana) were considered as the target population for the study. The students are actually practicing teachers drawn 
from all the ten educational districts in Botswana hence the target population has all the salient characteristics of the 
parent population. A total of 66 teachers (36 males and 30 females) were sampled; this repersents 13% of the target 
population 

4. Data Presentation and Analysis 

The questionnaire administered was divided into four sections: Section A focused on background information 
relating to the washback construct while Section B concentrated on classroom activities. Section C and Section D 
dealt with the influence of past examination questions on teaching and use of examination related booklets in the 
classroom respectively. 

4.1. Section A: Background Information 

Section A of the questionnaire had five questions based on examination washback hypotheses. Item 1 in this 
section collected opinions of teachers regarding the influence of examination on what teachers teach in the 
classroom. Fig. 3 below illustrates the distribution of responses alone a four point Likert scale categories. An 
overwhelming majority of 95.5% of the teachers responded positively to the statement. The responses of the teachers 
provide strong evidence to the hypothesis that examinations have a direct influence on the teaching and learning 
process. 



St ra ng, ly □ Isga re e □ isagree Agree St ro ng ly Ag re e 

Fig-2. Examinations have an influence on what teachers teach in the classroom 

Item 3 in the same section was designed to determine the relationship between content selection and examination 
practice. According to Fig. 2 60% of the teachers indicated that knowledge of the content of the examination 
influences their topic selection. However, 40% of the teachers said there was no link between selection of the topic 
and content of the examination. 
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Fig-3. Examinations will influence the teacher to select some topics for teaching 
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Items 4 of the scale asked the respondents to indicate the extent to which the examinations influence the degree 
or depth of teaching a given topic. Fig. 5 shows the distribution of responses. Out of 67 teachers who responded to 
the items, less than 10 of them (10.5%) disagreed This means a majority of the teachers were of the view that 
examinations influence the comprehensives teachers handle a topic. In other words, the amount of energy expended 
on a topic is determined by the content of the examination. 
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Fig-4. Examinations will influence the degree of depth of teaching a given topic 

4.2. Section B: Influence of Examinations on Classroom Activities 

Items in Section B of the questionnaire were designed to establish the extent to which examinations influence 
classroom activities. Table 6 below shows a list of questions and the pattern of responses. The distribution patterns 
are all skewed to the left indicating that majority of the teachers either agreed or strongly agreed with the statement 
presented. For example, close to 92.6% of the teachers indicate that selection of a teaching method is influenced by 
examinations. The evidence in Fig. 6 also shows that majority of teachers (80.6%) say that classroom activities 
revolve around examinations preparation. Also 62.7% of the respondents skip non-examinable topics in the syllabus. 
Also, responses to Item 16 points to a high likelihood that content designed to develop cognitive and affective 
domains of the learners is not taught because the content is not assessed. Exclusion of some topics from classroom 
activities means that important aspects of the syllabus are not taught. 

Table- 1. Frequencies indicating influence of examinations on classroom activates 


Classroom Activities 


Question 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

Q7 Selection of teaching methodology is 

influenced by the need to help students pass 

1.5 

6 

43.3 

49.3 

Q8 Do you normally cover the whole syllabus 

9 

38.8 

29.9 

22.4 

Q1 1 Classroom activities revolve around 
examinations preparation 

3 

16.4 

67.2 

13.4 

Q15 Teachers often skip content in the syllabus 

That is not examinable 

9 

28.4 

43.3 

19.4 

Q16 Content not examined is important for cognitive 
Development of the child 

1.5 

7.5 

50.7 

40.3 

Q20 There are objectives in the syllabus not assessed in the 
final examinations 

1.5 

6 

70.1 

22.4 


4.3. Section C: Influence of Past Examination Items on Learning and Teaching 

Two related items (i.e., Item 21 and Item 22) were used to measure the extent to which teachers use past 
examinations papers to prepare students for final examinations. Item 21 on the questionnaire asked teachers to 
indicate the extent to which items in the final examinations are repeated. A total of 85.1% of the teachers sampled 
agreed with the statement that items are often repeated. Item 22, on the other hand, required teachers to state whether 
they find past examinations papers to be a helpful teaching resource that can be used to prepare students for final 
examinations. Since the statement was put in a negative form (i.e., Past examinations questions are not helpful in 
preparing students for final examinations), over 70% of the teachers disagreed with the statement. The evidence 
provided by the two items indicates that teachers extensively use past examinations papers to prepare learners for the 
final examinations. Repetition of items over the years has the potential to engender negative washback effect on 
teaching since teachers will try to predict the kind of items that will appear on the examination. Teaching will as a 
result be focused on topics that are tested regularly. Important content that is not regularly tested will be ignored by 
the teachers. 

4.4. Section D: Use of Examinations Related Materials or Booklets 

Teachers often use examination related booklets together with the syllabus and other prescribed teaching 
materials. Fig. 9 below reflects items in Section D that measured the influence of examinations related booklets on 
teaching. 
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Table-2. Use of Examinations related booklets in the classroom 


Use of Examination Related Materials 


Questions 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

Q26 Teachers often use of Examinations related 

Booklets to teach their classes 

4.5% 

14.9% 

61.2% 

19.4% 

Q27 Examination related booklets cover most of 
the content in the syllabus 

0% 

26.9% 

59.7% 

13.4% 

Q28 Examination related booklets reproduce questions in 

examination 

1.5% 

20.9% 

68.7% 

9% 

Q29 Content in examinations related booklets 
encourage teaching to the test 

1.5% 

13.6% 

69.7% 

15.2% 

Q30 Content covered by examinations related booklets 
is linked to objectives in the syllabus 

1.5% 

9% 

74.6% 

14.9% 

Q32 Content in examinations related booklets is narrow 

7.5% 

50.7% 

35.8% 

6% 


The evidence presented in Table 9 clearly shows that teachers extensively use examinations related booklets 
since 80.6% of them agreed with item 26. Teachers rely on the booklets because they cover most of the information 
in the syllabus (Item 27) and reproduce examinations questions (Item 28). Therefore, it is not surprising for 85% of 
the teachers to say that examinations related booklets encourage teachers to teach to the test. Also, close to 45% of 
the teachers belief content in the booklets is narrow as compared to the subject matter presented in the syllabus. 

5. Qualitative Data Analysis 

Item 35 of the questionnaire required teachers to state whether they agree or disagree with the statement that: 
v Examinations related booklets can help improve the quality of teaching and learning in schools’. A total of 74.6% 
of the respondents agreed with the statement. An open ended part of the same question gave each respondent the 
opportunity to give reasons as to why they agree or disagree with the statement. Teachers give varying reasons in 
support of their answers; the following table show some of the themes that emerged from the narrative responses. 
The number in each column indicates the frequency of occurrence of each theme. For example, the Used for 
Revision theme was mentioned six times. Essentially, four main themes emerged and the themes are subsequently 
discussed. 



Themes Supporting the Statement 


Items in the Booklets 
are aligned 

To 

Final Examination 

Items 

Content in the 

Booklets 

is a summary of 
Syllabus Content 

Used for 
Revision 

Booklets are Based 
on Syllabus 

Objectives 

Booklets provide 

additional material and 
reduces boredom 

27 

9 

6 

6 

3 


Fig-5. Emerging themes for teachers who agreed with the statement 


5.1. Theme One: Booklets are aligned to the Final Examinations 

The most frequently stated reason was that items in the booklets are similar to the ones used in the final 
examinations. One teacher commented that: ‘This is because most the questions that can be found in the 
examination booklets can be found in an Examination question paper and these can help learners to be familiar with 
the type of questions that an examination can question ’. In other words, booklets address topics that are regularly 
tested. 

5.2. Theme Two: Booklets Summarize Content in the Syllabus 

Teachers have a feeling that examinations related booklets improve the quality of teaching and learning because 
they present content in a summary form. The subject matter in the booklets is not only condensed but it is also 
simplified. An interesting observation made by one teacher is that the booklets are useful: ‘Because most of the 
topics in the syllabus are covered in the booklets and are in a summary form, therefore they are lowered to every 
learners level’. Suffice to mention at this juncture that simplifying or lowering the difficulty of concepts in the 
syllabus might produce a big difference between the prescribes syllabus and what is actually taught in the classroom. 

5.3. Theme Three: The Booklets are used for Revision Purposes 

Since items in the booklets are similar to items used for final examinations, teachers often use the booklets to 
help learners prepare for the final examination. This perception is clearly captured in one comment made by a teacher 
to the effect that: ‘There is a lot of content and it ( examination related booklets ) makes it easy for revision purposes 
for learners and questions are shortened and aligned to the standards for PSLE hence improving the teaching and 
learning in schools and prepare learners for Exams ’. If items in the booklets target the same cognitive skills as final 
examinations questions, then the effect in the classroom will be positive. Teachers can use the items in the booklets 
to generate diagnostic information. Subsequent remedial exercises can be employed to address areas of weakness. 
However, shortening of an items may result in decontextualized question that tap on a different construct from the 
one targeted by the original item. Lack of correspondence between the skills assessed in the final examinations and 
those targeted by examinations booklet items creates room for negative washback effect. 
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5.4. Theme Four: The Booklets are based on Syllabus Objectives 

Comments made by the teachers indicate a strong link between content in the booklets and specific instructional 
objectives in the syllabus. One of the comments posted explained that teachers used the booklets because: 4 They 
cover most of the objectives in the syllabus and help teachers to identify testable and non testable objectives \ To the 
extent that booklets indicate testable and non-testable objectives, then teachers will concentrate on testable 
objectives. The non-testable objectives will be overlooked thus engendering negative washback effect. 

The general perception amongst teachers who have a favourable view on examination related booklets 
is that they are relevant teaching and learning materials. Although the content presented in the booklets 
might be limited, the subject matter is simplified to make it accessible to learners. The booklets also give 
students enough practice for the final examinations. Since 25.4% of the teachers sampled disagreed with the 
proposed idea that booklets contribute towards improvement of teaching and learning in schools; it is imperative to 
present their perspectives. These group of teachers presented ideas that contradicted opinions held by those who 
agreed. One teacher who disagreed stated that: v Teaching and learning is not about examination, it is equipping 
learners with skills and knowledge relevant to the world of work and be able to fit well in the community The view 
expressed by this teacher is consonant with the aims and objectives of basic education curriculum. One of the 
objectives is to help children develop a range of educational experiences that will enable them to discover and 
develop their own interests, talents and skills (Republic of Botswana National Commision on Education, 1993). 

6. Discussions 

The discussion of results will be based on the three research questions stated at the beginning of the paper. 
However, evidence provided by the participants indicates a strong overlap between the research questions. 

To what extent does examination influence the teacher’s content selection? 

Over 95% of the teachers who responded to the survey instrument indicated that examinations influence what 
they teach in the classroom and 60% were of the view that knowledge of examination questions influences their topic 
selection. Therefore, the subject matter assessed in the examination has a significant influence on what is taught in 
the classroom. One main reason may be the recycling of items from year to year thus making examination 
procedures to be predictable. Over 80% of the teachers agreed with the statement that examination items were often 
repeated. Repetition of items in the examination eventually leads teachers to select topics in the syllabus that are 
regular tested. Content in the syllabus that is not regularly tested is ignored; this point is consistent with the results of 
a study on examinations washback by Pizarro (2010) the researcher stated that: “Thus, the ET format leads teachers 
to ignore important aspects of the curriculum that are not tested in the examinations such as oral communication and 
interaction in the classroom’ (p. 164). The undesirable effect of this practice is that classroom activities are geared 
towards helping learners to memorize the items at the expense of mastery of syllabus content. 

To what extent does examination influence the degree or depth of teaching? 

A majority of the teachers surveyed were of the view that examinations influence the comprehensiveness 
teachers handle a topic. Teachers are able to concentrate more on examinable content because objectives that are not 
tested are skipped. Therefore, examinations have an influence in the degree or depth of teaching in the classroom by 
making teachers to spend less time on objectives that are not regularly tested. In the end, teachers put more effort and 
time on objectives that are regularly assessed in the final examinations. 

To what extent does examination influence the teacher’s selection of classroom activities? 

A large majority (92.6 %) of the teachers indicate that selection of a teaching method is influenced by 
examinations and over 80% say that classroom activities revolve around examinations preparation. This is further 
proof that past examination papers have a direct impact on the learning and teaching process. In this case, the impact 
is certainly negative because teachers only teach objectives that are regularly tested. The prescribed syllabus is 
effectively narrowed as most of the objectives are ignored. Another important finding extracted from the teachers’ 
narrative responses is use of examinations related materials or revision booklets in the classrooms. Examination 
revision booklets are a compilation of past papers produced by private publishers. The use of such booklets has 
increased in schools because teachers perceive them as being relevant content sources. However, the booklets have 
two serious limitations that have the potential to impact negatively on teaching and learning; these are narrowing 
down of the content in the syllabus and simplification of concepts. 

The examination revision booklets only address objectives in the syllabus that regularly appear in the final 
examinations; non-testable objectives are left out. Some objectives might not be testable but they have a central role 
to play towards provision of a holistic education; that is, education that develops the learners cognitive and affective 
domains. Teachers also indicated that the subject matter in the booklets has been simplified to suite the level of the 
learners. Objectives in the syllabus generally fall into two groups; high order and low order objectives. Simplifying 
objectives in the syllabus means using low order questions to assess application, analysis and synthesis skills; this 
will create a mismatch between the objective in the syllabus and the item in the test. Objectives that targets high 
order thinking skills are included in the syllabus as a way of preparing students for higher levels of education. If such 
objectives are not tested directly then there will be poor articulation between primary and junior secondary school 
levels. Students proceeding to junior secondary schools will undoubtedly perceive the content to be too difficult 
because they lack the necessary foundation. The evidence generally supports the hypothesis that examinations will 
influence what teachers teach as well as the degree and depth of teaching (Alderson and Wall, 1993). 

7. Conclusion 

The quantitative and qualitative analysis of data from a sample of teachers has pointed towards the possible 
existence of negative examinations effect in the learning and teaching process. The most fundamental negative effect 
of examination on learning and teaching appear to be narrowing down of the syllabus content. This results from 
inadequate sampling of the domain during test construction and the tendency for items to be recycled. According to 
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Messick (1996) narrowing down of the content results “In the threat to validity known as construct 
underrepresentation (which jeopardizes authenticity), the assessment is deficient: The test is too narrow and fails to 
include important dimensions or facets of focal constructs” (p. 7). Also, examinations booklets used in schools are 
sources of negative washback largely due to simplification of content and promotion of teaching to the test. The 
apparent negative washback effect is working against the intents and purposes of criterion testing system in 
Botswana. Construct underrepresentation (due to inadequate sampling) and construct irrelevant variance (due to 
simplification of high order objectives) affects the validity of the examination. In the end, the examination will fail to 
reveal what the learner knows, understands and can do in relation to the content universe. Grading of learners will 
revert to using normative interpretation of scores rather content mastery standards as demanded by CRT. There is 
need to develop an assessment practice that will compel teachers and learners to concentrate on the subject matter as 
presented in the prescribed syllabus so that the major benefits of CRT as stated in the Revised National Commission 
on Education of 1993 can be realized. 
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